The impact of item-writing flaws and item complexity on examination item difficulty and discrimination value

نویسندگان

  • Bonnie R. Rush
  • David C. Rankin
  • Brad J. White
چکیده

BACKGROUND Failure to adhere to standard item-writing guidelines may render examination questions easier or more difficult than intended. Item complexity describes the cognitive skill level required to obtain a correct answer. Higher cognitive examination items promote critical thinking and are recommended to prepare students for clinical training. This study evaluated faculty-authored examinations to determine the impact of item-writing flaws and item complexity on the difficulty and discrimination value of examination items used to assess third year veterinary students. METHODS The impact of item-writing flaws and item complexity (cognitive level I-V) on examination item difficulty and discrimination value was evaluated on 1925 examination items prepared by clinical faculty for third year veterinary students. RESULTS The mean (± SE) percent correct (83.3 % ± 17.5) was consistent with target values in professional education, and the mean discrimination index (0.18 ± 0.17) was slightly lower than recommended (0.20). More than one item-writing flaw was identified in 37.3 % of questions. The most common item-writing flaws were awkward stem structure, implausible distractors, longest response is correct, and responses are series of true-false statements. Higher cognitive skills (complexity level III-IV) were required to correctly answer 38.4 % of examination items. As item complexity increased, item difficulty and discrimination values increased. The probability of writing discriminating, difficult examination items decreased when implausible distractors and all of the above were used, and increased if the distractors were comprised of a series of true/false statements. Items with four distractors were not more difficult or discriminating than items with three distractors. CONCLUSION Preparation of examination questions targeting higher cognitive levels will increase the likelihood of constructing discriminating items. Use of implausible distractors to complete a five-option multiple choice question does not strengthen the discrimination value.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Reviewing the results of qualitative and quantitative analysis of MCQs in Introduction to clinical medicine course

Background: At most of the medical universities, MCQ-based examinations are often use as the first component, particularly to ensure the candidates have an adequate knowledge base, prior to entering subsequent clinical examinations. Method: In ICM , 3973 MCQs were collected from tests over a four-year period from 2005 to2009. Questions were evaluated for 10 frequently occurring item writing f...

متن کامل

Psychometric Properties of 3-, 4-, and 5-Option Item Tests: Do Test Takers’ Personality Traits Make a Difference?

Prior research has yielded mixed results regarding what contributes to psychometrically sound multiple-choice (MC) items. The purpose of the present study was, therefore, twofold: (a) to compare 3-, 4-, and 5-option multiple-choice (MC) tests in terms of psychometric characteristics, and (b) to investigate the relationships between three MC tests and five personality traits. To that end, 150 st...

متن کامل

Does number of options in multiple choice tests affect item facility and discrimination? An examination of test-taker preferences

Multiple Choice tests are utilized widely in educational assessment because of their objectivity, ease of scoring, and reliability. This study aimed to compare IF and ID of MC vocabulary test items and attempted to find whether these indices are affected by the number of options. To this end, four 20 item stem equivalent vocabulary tests (3-, 4-, 5-, and 6-option MC) were administered to 194 (1...

متن کامل

افت در پارامترهای سؤال: مفاهیم، روش‌شناسی و شناسایی

Item Parameter Drift occurs over time for various reasons; when test items lose their initial characteristics, such as difficulty and discrimination parameters. Including cases of item parameter drift are revealed, excessive repetition, changes in the education system, and the position of items and the parameters of poor initialization. Item parameter drift causes of the invariance to be violat...

متن کامل

Utility of Complex Alternatives in Multiple-Choice Items: The Case of All of the Above

This study investigated the utility of all of the above (AOTA) as a test option in multiple-choice items. It aimed at estimating item fit, item difficulty, item discrimination, and guess factor of such a choice. Five reading passages of the Key English Test (KET, 2010) were adapted. The test was reconstructed in 2 parallel forms: Test 1 did not include the abovementioned alternative, whereas Te...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:

دوره 16  شماره 

صفحات  -

تاریخ انتشار 2016